There has been a research for a technology to detect a specific retrieval word to extract a part related to desired contents from voice data obtained by recording telephone calls or recording voices in lectures or conferences. In such a voice retrieval technology, there has been proposed a technology to speed up retrieval processing (see, for example, Japanese Laid-open Patent Publication No. 2002-278579). A voice data retrieval device disclosed in Japanese Laid-open Patent Publication No. 2002-278579, for example, converts voice waveform data into a voice symbol string and detects a section that matches with a voice symbol of a retrieval word in the voice symbol string. Then, the voice data retrieval device uses the voice waveform data in the detected section to determine whether or not the retrieval word is included in the section.